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1.  Three  Technical  Papers.  ~3Bart  wrote  three  full-length  technical  papers 
during  Qtr  3:  ^Fuzzy  Associative  Memories?5'  ^Hidden  Patterns  In  Combined 
Knowledge  Networks,9^  and,  with  Clark  Guest,  ^Optical  Bidirectional 
^SS?C'* ativLe^Memorl es^^  Preprints  of  these  papers  were  sent  to  AFOSR  and 
DARPA. 


"Fuzzy  Associative  Memories"  has  been  accepted  as  a  chapter  In  Fuzzy 

Expert  Systems,  edited  by  Abraham  Kandel,  to  appear  as  an  Addlson-Wesley 

book  In  1987.  This  paper  contains  the  results  on  FAMs  reported  In  the  first 

quarterly  R&D  status  report.  It  also  contains  new  theorems  on  fuzzy 

foundations.  Perhaps  the  most  Important  result  Is  that  several  popular 

continuous  associative  memories— Cohen-Grossberg,  Hopfleld,  BAM,  Anderson 

braln-state-ln-a-box,  Grossberg  competitive  learning— all  behave  In  their 

distributed  computations  so  as  to  minimize  fuzzy  entropy.  Information  is 

n 

stored  as  fixed  points  at  the  vertices  of  the  unit  hypercube  [0,  1]  .  An 
ambiguous  Input  enters  the  cube  as  a  point  (fuzzy  set)  in  its  Interior  or  on 
Its  surface.  As  the  distributed  computation  proceeds,  the  Input  Is 
disambiguated  as  the  point  moves  toward  the  nearest  (Boolean)  vertex. 

"Hidden  Patterns  In  Combined  Knowledge  Networks"  is  In  review  at  the 
International  Journal  of  Approximate  Reasoning.  It  discusses  the  fuzzy 
cognitive  map  (FCM)  knowledge  combination  technique  developed  during  the 
first  quarter  of  OCCAM.  The  paper  also  contains  results  on  FCM  limit-cycle 


Diot 


o^cial 


1  /  or 


i 


behavior. 


"Optical  Bidirectional  Associative  Memories'*  will  appear  In  the  SPIE 
Proceedings:  Image  Understanding,  vol.  758,  In  1987.  A  more  general 

version  of  the  paper  Is  In  preparation;  It  will  be  submitted  to  Optical 
Engineering.  This  papers  summarizes  the  theorems  on  discrete  and  continuous 
BAMs  developed  during  the  first  two  quarters  of  OCCAM.  Four  new  BAM  optical 
Implementations  are  described  that  emphasize  a  spatial  light  modulator, 
laser  diodes  and  high-speed  detectors,  a  reflection  hologram,  and  a 
transmission  hologram. 

2.  Fuzzy  Cognitive  Map  Limit  Cycle  Experiments.  Limit  cycles  were  studied 
In  simple  binary  FCMs  to  see  whether  approximate  symmetry  of  matrix 
shortened  limit  cycles  or  decreased  their  frequency  of  occurrence.  We  were 
partlculary  Interested  In  characteristics  of  FCMs  that  posssess  short  limit 
cycles.  Limit  cycles  (points)  of  length  1  In  FCMs  are  interpreted  as 
unequivocal  predictions. 

We  hypothesized  that  high  degrees  of  sparseness  or  symmetry  of  FCM 

connection  matrices  should  often  be  associated  with  short  limit  cycles.  For 

example.  If  an  entire  column  of  an  FCM  matrix  is  zero,  then  the  activation 

of  the  corresponding  concept  node  cannot  change.  If  M  of  the  N  concept 

nodes  cannot  change  activation,  then  the  maximum  possible  limit  cycle 

M-N 

length  (for  synchronous  operation)  drops  by  a  factor  of  2  ,  and  actual 

lengths  should  be  expected  to  drop  accordingly. 

We  computed  all  limit  cycles  for  tractable-sized  FCM  matrices  with 
randomly  chosen  connection  strengths,  and  tried  to  relate  short  limit  cycles 
to  sparseness,  symmetry,  or  other  matrix  properties. 

We  also  Investigated  other  limit  cycle  properties  besides  length,  such 


N 

as  the  number  of  different  limit  cycles  and  the  number  of  (the  2  possible) 

Initial  states  that  resulted  In  a  given  limit  cycle. 

Sparseness  was  measured  In  two  ways:  by  the  ratio  fo  nonzero  columns 

to  total  columns,  and  by  the  ratio  of  nonzero  elements  to  total  elements. 

Symmetry  was  measured  by  the  ratio  S/(S+D)  of  the  sum  norm  S  of  the 

matrix  to  the  sum  of  the  sum  norm  and  difference  norm  D  of  the  matrix.  Here 

the  sum  norm  of  a  matrix  Is  defined  as  the  norm  of  the  sum  of  the  matrix  and 

Its  transpose.  The  difference  norm  Is  defined  as  the  norm  of  the  difference 

1 

of  the  matrix  and  Its  transpose.  The  matrix  norm  we  used  was  the  1  norm, 
the  sum  of  the  absolute  values  of  the  matrix  entries.  A  little  thought 
shows  that  this  symmetry  Is  unity  for  a  symmetric  matrix  and  zero  for  an 
anti -symmetric  (real  Hermltlan)  matrix.  Such  a  symmetry  measure  therefore 
Is  not  totally  satlsfactor  for  anti-symmetry  is  In  some  sense  highly 
symmetric. 

Consequently,  no  strong  tendency  was  shown  between  matrices  with  large 
symmetry  ratios  and  short  limit  cycles,  though  many  suggestive  cases  did 
arise.  Sparseness  also  proved  largely  uncorrelated  with  our  sparseness 
measure. 

The  most  striking  result  of  the  experiment  was  that  "zero-symmetric" 

matrices  only  have  fixed  points.  Here  we  define  zero-symmetry  as  follows: 

If  f  Is  nonzero,  then  f  *  0,  We  studies  zero-symmetric  10-by-10 
1j  ji  _  ^ 

matrices  with  uniformly  random  Integer  entries  in  •••  »  100y  or  real 

entries  In  [-1,  1]  (with  uniformly  varying  sparseness).  Only  1  out  of  15 

matrices  had  a  limit  cycle  not  of  length  1;  this  could  have  been  due  to  the 

synchronous  operation  (e.g.,  even  Hopfleld/Cohen-Grossberg  symmetric 

autoassoclators  exhibit  small  limit-cycles  in  synchronous  operation). 

Decreased  sparseness  made  some  longer  limit  cycles  somewhat  more  likely.  In 


•  FCM  context*  zero-symmetry  means  unidirectional  causal  flow*  which  does 
not  seem  a  significant  restriction  considering  the  resulting  gain  of  usually 
unanimous  expert  opinion. 

Finally*  sparseness  turned  out  not  correlated  with  limit  cycle  length 
but  with  limit  cycle  number.  Indeed,  very  sparse  matrices  seemed  to  have 
many  short  limit  cycles  and  very  dense  matrices  seemed  to  have  fewer  limit 
cycles. 

3.  DARPA  Program  Review.  Bart  presented  an  OCCAM  briefing  on  11  February 
1987  to  DARPA 's  (AFOSR's)  Optical  Computing  Program. 


II.  OCCAM  at  UCSD 

During  Qtr  3  of  the  OCCAM  Program  at  UCSD  the  systems  work  begun  in  the 
second  quarter  provided  Its  Initial  research  results.  This  report  reviews 
results  obtained  In  the  areas  of  an  all-optical  Implementation  of  the 
continuous  symmetric  unidirectional  Cohen-Grossberg  autoassoclator  model 
(which  subsumes  the  Hopfleld  model),  neural  net  Implementations  of  sclase 
and  rotation  Invariance,  and  pattern  correlation  In  photorefractlve 
crystals. 

Before  proceeding  Iwth  review  of  specific  results.  It  Is  Important  to 
review  guiding  concepts  that  have  emerged  to  date  through  OCCAM.  The 
primary  charter  of  OCCAM  Is  to  Implement  cognitive  processing  or  conceptual 
computing  on  optical  systems.  The  Bidirectional  Associative  Memory  (BAM) 


has  been  Identified  as  a  promising  network  paradigm  to  carry  out  much  of 
this  charter.  Not  only  has  the  BAN  been  shown  to  have  Important 
architectural  properties  such  as  heteroassociativity,  guaranteed  convergence 
for  pattern  recognition  and  completion,  and  stable  limit  cycles  for  temporal 
coding.  It  also  naturally  lends  Itself  quite  naturally  to  optical 
Implementations  (see  "Optical  BAMs,"  discussed  above).  The  bidirectional 
optical  connectivity  provided  by  matrix  masks  or  holograms  can  be  directly 
used. 

An  additional  attractive  feature  of  the  BAH  is  the  way  Its  basic 
architecture  can  be  extended  In  steps  toward  the  features  of  a  full  adaptive 
resonance  theory  (ART)  model,  as  discussed  In  the  OCCAM  Phase  II  Proposal. 
A  powerful  synergism  has  been  Identified  between  optical  Implementations  of 
fuzzy  logic  operations,  which  have  been  studied  In  OCCAM,  and  the 
competitive  self-normalizing  fields  of  ART.  Both  systems  require  essentially 
a  maximum  (minimum)  operation  over  a  field  fo  elements.  It  appears  likely 
that  a  single  optical  Implementation  will  serve  these  purposes. 

A  second  charter  for  OCCAM  has  been  the  specification  of  an  operating 
system  for  optical  associative  memories.  Operating  systems  perform  two 
functions:  Interface  of  a  processor  to  the  outside  world,  and  task 
sequencing  or  goal  orientation,  the  problem  of  Interface  has  been  studied 
In  OCCAM  Phase  I  (goal  orientation  has  been  scheduled  for  study  during  OCCAM 
Phase  II).  the  specific  problem  of  where  traditional  preprocessing  should 
leave  off  and  neural  processing  should  commence  has  been  of  particular 
Interest.  It  now  appears  that  the  dividing  line  Is  not  as  sharp  as  first 
believed.  Though  traditional  Implementations  predominate  In  processing  raw 
real-world  data,  benefit  can  be  drived  by  Incorporating  some  neural 
processing  functions.  Likewise,  neural  processing  predominates  at  the 


abstract  level,  but  some  traditional  methods  may  still  be  of  use. 

An  all  optical  system  for  Implementing  the  continuous  Cohen-Grossberg 
meodel  was  compleded  during  Qtr  3.  This  system  may  be  viewed  as  a  subset, 
or  simplification,  of  a  full  continuous  BAM  that  will  eventually  be 
Implemented.  Initial  findings  from  the  system  are  that  characteristics  of 
the  threshold  function  curve  are  rather  critical  to  acceptable  system 
operation.  The  displacement  of  high  gain  to  moderately  high  Input  values 
and  the  magnitude  of  the  saturation  gain  appear  to  be  particularly 
Important.  A  theoretical  and  computer  simulation  study  has  been  commenced 
to  quantify  the  effects  of  threshold  parameters  on  system  operation. 

For  the  purpose  of  comparing  traditional  methods  of  scale  and  rotation 
Invariant  Image  recognition  to  neural  Implementations  of  the  same,  work 
began  on  computer  modelling  of  a  "backpropagation"  network  trained  for 
Invariant  recognition.  The  model  Is  trained  on  the  letters  "A"  and  "T"  in  a 
number  of  different  rotations.  It  then  Is  able  to  classify  Input  "A"  and 
"T"  patterns  presented  at  those  and  similar  angles.  Work  Is  proceeding  to 
determine  If  It  Is  possible  to  train  the  network  for  geneneral  rotation 
Invariant  recognition.  That  Is,  if  It  Is  trained  to  be  rotation  invariant 
for  "A"  and  "T"  patterns,  will  It  exhibit  Invariance  for  other  untrained 
patterns? 

In  the  area  of  application  of  photorefractlve  materials  to  neural  nets, 
one  project  was  completed  and  another  begun.  Work  begun  during  the  second 
quarter  on  two-wave  mixing  In  a  reflection  configuration  culminated  during 
the  third  quarter  with  a  demonstration  of  correlation  of  pattern  images 
using  barium  tltlnate  In  a  reflective  pumping  arrangement.  Images  were 
brought  In  from  opposite  sides  of  the  crystal,  one  Image  weak,  and  one 
strong.  Power  in  the  weak  Image  was  amplified  In  proportion  to  its 


correlation  (overlap)  with  the  strong  Image.  Maximum  amplification  factors 
of  60  or  greater  were  achieved.  This  correlation  operation  can  be  expected 
to  prove  useful  In  pattern  Identification  for  optical  neural  networks. 

Work  started  In  Qtr  3  on  holographic  association  of  phase  encoded 
Images  In  photorefractlve  materials.  Previous  methods  of  association  have 
used  angular  multiplexing  of  Images  and  fractal  decimation  of  associated 
pixels  to  prevent  Bragg-cone  degeneracy  crosstalk.  Only  phase  encoding  of 
Images  provides  full  Image  heteroassociation  and  low  crosstalk. 
Pseudorandom  computer  generated  phase  masks  will  be  used,  and  lithium 
nlobate  Is  the  photorefractlve  that  will  Intlally  be  studied. 


